More cat than cute?: interpretable prediction of adjective-noun pairs

摘要

The increasing availability of affect-rich multimedia resources has\udbolstered interest in understanding sentiment and emotions in and\udfrom visual content. Adjective-noun pairs (ANP) are a popular midlevel\udsemantic construct for capturing affect via visually detectable\udconcepts such as “cute dog" or “beautiful landscape". Current stateof-\udthe-art methods approach ANP prediction by considering each\udof these compound concepts as individual tokens, ignoring the underlying\udrelationships in ANPs. This work aims at disentangling\udthe contributions of the ‘adjectives’ and ‘nouns’ in the visual prediction\udof ANPs. Two specialised classifiers, one trained for detecting\udadjectives and another for nouns, are fused to predict 553 different\udANPs. The resulting ANP prediction model is more interpretable\udas it allows us to study contributions of the adjective and noun\udcomponents.